Stereo and multi-view imaging has a long and rich history stretching back to the early days of photography. Stereo cameras employ multiple lenses to capture two images, typically from points of view that are horizontally displaced, to represent the scene from two different points of view. Such image pairs are displayed to the left and right eyes of a human viewer to allow the viewer experience an impression of three dimensions (3D). The human visual system can merge information from the pair of properly constructed images to achieve the perception of depth.
Stereo cameras can come in any number of configurations. For example, a lens and a sensor unit can be attached to a port on a traditional single-view digital camera to enable the camera to capture two images from slightly different points of view, as described in U.S. Pat. No. 7,102,686 to Orimoto et al., entitled “Image-capturing apparatus having multiple image capturing units.” In this configuration, the lenses and sensors of each unit are similar and enable the interchangeability of parts. U.S. Patent Application Publication 2008/0218611 to Parulski et al., entitled “Method and apparatus for operating a dual lens camera to augment an image,” discloses another camera configuration having two lenses and image sensors that can be used to produce stereo images.
Further, it is known that stereo pairs can be constructed in other ways. For example, with a standard single-view camera, two horizontally displaced images can be captured to represent the left- and right-views that the two eyes of a human would see. Still further, a stereo pair of images can be constructed from a single image by first defining the three dimensional geometry of the scene (e.g., as taught by Saxena et al. in the article “Make3D: Learning 3D scene structure from a single still image,” published in IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 31, pp. 824-840, 2009) Then, standard computer graphics techniques can be used to produce a textured rendering for the left- and right-eyes, using the three-dimensional model as the model and the textures from the single image.
Still further, methods of producing stereo images from videos have also been explored. For example, when the camera pans horizontally when capturing a video of a static scene, pairs of frames including one frame from the video and a second delayed frame from the same video (e.g., a delay of 1/10 seconds) can be combined to produce an acceptable stereo image. This is described in the article “Three dimensional video for the home,” by Op de Beeck et al. (Proc. International Conference on Augmented, Virtual Environments and Three-Dimensional Imaging, pp. 188-191, 2001). A similar phenomena is exercised by the well-known Pulfrich effect, where a single dark lens is worn in a pair of spectacles. The dark lens delays the human visual system's processing of the image stream for that eye, effectively producing stereo pairs of images by having one eye see a delayed version of the video.
In another line of teaching, J. S. McVeigh et al, in the article “Algorithm for automated eye strain reduction in real stereoscopic images and sequences,” (Human Vision and Electronic Imaging, Vol. 2657, pp. 305-316, 1996) describe a method for reducing eye strain in stereoscopic image viewing. The method describes a statistical model that can measure disparity range and force all points to lie on, or behind the plane of display screen so as to reduce eyestrain. However, it provides no systematic way to automatically improve the perception of 3D in a stereo image by using a model that estimates the quality of the stereo image.
U.S. Pat. No. 6,927,769 to Roche et al., entitled “Stereoscopic image processing on a computer system,” discloses a system for producing a stereoscopic image. The system permits a user to load two images into a stereoscopic image pair. In addition it permits a user load a single 3D image to be loaded into a computer to be morphed into a stereoscopic image pair.
U.S. Pat. No. 6,191,809 to Lee et al., entitled “Method of and apparatus for rectifying a stereoscopic image” discloses a method of rectifying a stereoscopic image consisting of left and right captured images comprises determining left and right rectification transformations. The goal of transformation is to minimize vertical disparity between the left and right captured images.
U.S. Pat. No. 4,472,037 to Lipton et al., entitled “Additive color means for the calibration of stereoscopic projection” discloses a method for calibrating right and left image views for both equal and even illumination intensity.